Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify interpreter and witness generation logic. #56

Merged
merged 16 commits into from
Feb 29, 2024

Conversation

LindaGuiga
Copy link
Contributor

This PR aims at unifying the logic of the interpreter and that of the CPU, for two main reasons:

  • the interpreter is now used by jumpdest_analysis during generation,
  • the continuation will also need the interpreter to generate segments.

The discrepancy between the interpreter and CPU logic has led to some bugs in those two use cases, and the unification would make the debugging and usage much easier.

Another change in the Interpreter is the removal of the lifetime:

  • prover_inputs are already available in the generation_state, so we use these instead of the Interpreter's specific field
  • since all tests -- except one small tests only aimed at testing the interpreter run -- use the kernel code, this PR always assumes that the Interpreter's code is the kernel code, which means we can also remove the code field of the Interpreter (which also required a lifetime).

Since one of the motivations for this PR is to prepare for continuation, it also includes the following changes, which make segment generation much easier:

  • the values in a MemoryState are stored as Option<U256>,
  • the initial MPT values are not stored in the TrieData segment if we are in the interpreter, but instead stored in the new interpreter field preinitialized_segments.

Those two changes make it easier to determine accurately which parts of a memory have been accessed during one segment execution.

Comment on lines +492 to +499
let get_vals = |opt_vals: &[Option<U256>]| {
opt_vals
.iter()
.map(|&elt| match elt {
Some(val) => val,
None => U256::zero(),
})
.collect::<Vec<U256>>()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of this stuff going on in the PR, a consequence of using Option<U256>. Are we sure we can't avoid wrapping values in Option? Can we just use 0 or U256::MAX or some other trickery?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we need to know whether a certain memory spot was set or not. But memory values can take any U256 value, so unfortunately, I think we do need the Option<U256>...

Copy link
Contributor

@dvdplm dvdplm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(partial review)

@4l0n50
Copy link
Contributor

4l0n50 commented Feb 23, 2024

PR always assumes that the Interpreter's code is the kernel code, which means we can also remove the code field of the Interpreter (which also required a lifetime).

But ad-hoc code might be useful for testing macross or any code not in the kernel.

@dvdplm
Copy link
Contributor

dvdplm commented Feb 23, 2024

But ad-hoc code might be useful for testing macross or any code not in the kernel.

Maybe we can revisit this when we have a concrete need for it? Or is this a blocker in your opinion?

@4l0n50
Copy link
Contributor

4l0n50 commented Feb 23, 2024

But ad-hoc code might be useful for testing macross or any code not in the kernel.

Maybe we can revisit this when we have a concrete need for it? Or is this a blocker in your opinion?

I'm not strong about this. The only problem could be that when we have this test (I have something like that on another branch) we would need to come back to the previous more general version of the interpreter where the code can be anything. But maybe there's an easy trick that I can't see now (here's the test)

Copy link
Contributor

@hratoanina hratoanina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass.
I'll take another look later.

}
}

/// Returns the value of a `State`'s clock.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not in this PR, but we should unify the clock someday. I think we should add a clock register for the prover too, but it's not a strong opinion either.

@Nashtare Nashtare added this to the zk-continuations - Q1 2024 milestone Feb 27, 2024
LindaGuiga and others added 3 commits February 27, 2024 16:08
* Add traits State and Transition

* Move perform_state_op to Transition

* Fix bug

* Get rid of is_generation_state + add reviews

* Remove TDO

* Fix handle_error

* Move preinitialized_segments from the interpreter to MemoryState

* Clippy

* Apply get_halt_context comment

---------

Co-authored-by: Linda Guiga <lindaguiga3@gmail.com>
Copy link
Collaborator

@Nashtare Nashtare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Linda! I may need to get back to it one more time, but leaving some comments in the meantime for discussion

@@ -87,7 +87,7 @@ fn test_jumpdest_analysis() -> Result<()> {
.get_mut(&CONTEXT)
.unwrap()
.pop();
interpreter.push(U256::one());
interpreter.push(41.into());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it changed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the unification, the proofs are checked in the reverse order. I'm still not sure why, but it must come from a previous discrepancy between the interpreter and witness generation that we hadn't identified...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @4l0n50 in case you have an explanation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. Maybe is because the jumpdests where sorted at some point but is not done anymore? (as it was no necessary)

Copy link
Collaborator

@Nashtare Nashtare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Linda! Looks good to me!

@@ -432,7 +483,7 @@ impl<F: Field> State<F> for GenerationState<F> {
fn incr_interpreter_clock(&mut self) {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd rather have the trait define the blanket no-op impl, so that we don't have to keep a reference to something that's interpreter specific here.

Copy link
Contributor Author

@LindaGuiga LindaGuiga Feb 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, in the end, I think it's better to increment the interpreter clock in push_cpu, since this is when the generation "clock" is increased. This would get rid of that method altogether, and would insure a better unification (we noticed that the clock of the interpreter was not increased when calling generate_exception)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah it's even better

@LindaGuiga LindaGuiga merged commit 9252d0e into 0xPolygonZero:main Feb 29, 2024
5 checks passed
@Nashtare Nashtare deleted the interpreter-unified branch March 6, 2024 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants